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1  Research  Overview 


The  core  of  my  proposal  to  ONR  was  the  following  sentence;  “The  concrete 
goal  of  the  proposed  research  is  the  design  and  implementation  of  a  modem 
environment  for  interactive  data  analysis.’’  The  extent  to  which  I  have 
succeeded  in  this  goal  is  represented  by  the  current  state  of  a  system  called 
Arizona,  which  is  described  in  detail  in  [13,  12).  Arizona  is  the  latest  in  a 
series  of  “systems”  which  have  been  the  focus  of  my  research  for  the  past 
three  years.  Each  of  the  systems  before  Arizona  investigated  a  particular 
aspect  of  a  comprehensive  data  analysis  environment;  the  purpose  of  Arizona 
is  to  put  them  together.  In  brief,  the  systems  were: 


•  Antelope:  experimentation  with  object-oriented  databases  and  constraint- 
based  interactive  graphics. 


1 


•  Fossil:  interfaces  to  traditional  (Fortran)  scientific  subroutine  libraries. 


•  Ceu:tus;  a  mathematician’s  numerical  linear  algebra  package,  using 
object-oriented  programming  to  provide  a  higher  level  of  abstraction 
than  Linpack[6]. 


1.1  Arizona 


Arizona  is  intended  to  be  a  portable,  public-domain  collection  of  tools  sup¬ 
porting  scientific  computing, ^j^uanlitative  graphics,  and  data  analysis,  im¬ 
plemented  in  Common  Lisp^2,  CLOS  (the  Common  Lisp  Object 

System)[3,  iC^AJthough  thefFITiubstantial  implementation  of  some  of  the 
modules  described  below,  more  years  of  work  are  required  before  it  will  ma¬ 
ture  and  stabilize  to  the  point  of  robust  production-quality  code\  Arizona 
is  intended  primarily  as  a  research  vehicle;  however,  I  hope  that 
embodied  in  its  design  are  of  interest  in  themselves  and  of  use  in  future 
scientific  computing  and  data  analysis  systems  (eg.  a  “New  New  S’’[2]). 


Arizona  evolved  out  of  the  Fossil  meeting  discussed  below.  The  code  in 
the  current  version  of  Arizona  is  perhaps  80%  my  work,  with  the  remainder 
contributed  by  John  Michalak,  Michael  Sannella,  Werner  Stuetzle,  Robert 
Gentleman,  Catherine  Hurley,  and  .4ndrew  Bruce  of  the  U.  of  Washington, 
Jan  Pedersen  of  Stanford  University  and  Xerox  PARC,  Steve  Peters  of  MIT, 
UC  Berkeley,  and  Apple  Computers,  and  Wayne  Oldford  of  MIT  and  U.  of 
Waterloo.  In  addition,  code  from  Arizona  has  been  distributed  to  and  is 
the  subject  of  continuing  collaboration  with  researchers  at  U.  Minnesota, 
Carnegie  Mellon,  Bell  Labs,  and  Bellcore. 

Discussion  of  the  philosophy  underlying  Arizona  can  be  found  in  [14, 15, 
11,  12,  16,  23].  Briefly,  the  design  is  motivated  by  the  belief  that  an  ideal 
system  for  scientific  computing  and  data  analysis  should  have: 


•  One  language  that  can  be  used  for  both  line-by-line  interaction  and 
defining  compiled  procedures. 

•  Minimal  overhead  in  adding  new  compiled  procedures  (or  other  defi¬ 
nitions). 
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•  language  that  supports  a  wide  variety  of  abstractions  and  the  defi¬ 
nition  of  new  kinds  of  abstraction^ 

Cj^Programming  tools  (editor,  debugger,  browsers,  metering  and  moni¬ 
toring  toolsj,. 

•  AutomatisjB^ory  management  (dynamic  space  allocation  and  garbage 
,  collection). 

f 

^Portability  over  many  types  of  workstations  and  operating  systems. 

r  O  ^  %  • 

•  A  community  of  users  and  developers. 


•  Access  to  traditional  Fortran  scientific  subroutine  libraries  or  equiva¬ 
lents. 


'it 


•  A  representation  of  scientific  data  directly  in  the  data  structures  of 
the  language. 

•  Comprehensive  numerical,  graphical,  and  statistical  functionality. 

•  Device  independent  static  output  graphics. 

•  Window  based  interactive  graphics. 

•  Support  for  efficient  and  concurrent  access  to  large  databases. 

•  Documentation  and  tutorials,  both  paper  and  on-line. 


The  first  nine  points  (through  “access  to  Fortran”)  come  for  free  with  stan¬ 
dard  Common  Lisp  environments.  The  remaining  six  are  the  research  as¬ 
pects  of  Arizona. 

Arizona  is  divided  into  a  number  of  modules,  with  limited  interdepen¬ 
dencies,  to  permit  individual  modules  to  stabilize  and  be  “released”  before 
the  whole  system  is  complete. 

The  modules  are  divided  into  two  groups:  a  numerical,  quantitative 
kernel  and  an  interactive,  window-based,  scientific  graphics  part. 
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1.1.1  A  quantitative  kernel 


The  non-graphical  quantitative  kernel  is  more  developed  at  present,  because 
it  can  be  implemented  in  an  efficient,  portable  way  using  existing  standards 
for  Common  Lisp  and  CLOS.  The  quantitative  kernel  consists  of; 


•  Basic  Math,  which  requires  Common  Lisp, 

•  Collections,  which  requires  Common  Lisp  and  CLOS, 

•  Linear  Algebra,  which  requires  Basic  Math  and  Collections  (sometimes 
this  module  is  refered  to  as  Cactus), 

•  Probability,  which  requires  Linear  Algebra, 

•  Database,  which  requires  Collections,  and 

•  Statistics,  which  requires  Database  and  Probability. 

The  contents  of  these  modules  are  described  in  [13].  The  Linear  Algebra 
module  is  the  Cactus  system  discussed  below. 


1.1.2  Static  output'only  graphics 

Since  [13]  was  written,  static,  output-only  graphics  has  been  added  to  Ari¬ 
zona  using  two  modules,  which  are  based  on  the  Xerox  Lisp  PLOT  package 
designed  and  written  by  Jan  Pedersen[17]; 


•  2D~Graphics,  which  is  essentially  a  device  driver  for  various  Common 
Lisps  and  displays.  It  defines  a  Graphics-Canvas  abstraction  which 
provides  the  ability  to  draw  points,  lines,  polylines,  bitmaps,  char¬ 
acters,  strings,  etc.,  in  device  coordinates.  Implementations  exist  for 
Xerox  Common  Lisp,  Symbolics  Genera  Windows,  and  Coral  Com¬ 
mon  Lisp  (on  the  Apple  Mac  II).  Implementations  for  X  windows  and 
Postscript  printers  are  planned  for  the  near  future. 
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•  2D-Plot,  which  provides  higher-level  scientific  graphics,  such  as  scat- 
terplots,  histograms,  boxplots,  etc.  The  basic  abstraction  is  the  Plot, 
which  is  a  hierarchy  of  Plot-Objects  (points,  curves,  strings,  etc.)  that 
are  defined  in  the  user  or  world  coordinate  system.  A  Plot  can  be 
drawn  on  any  available  Graphics-Canvas  (eg.  a,  window  or  a  printer); 
the  transformation  from  world  to  device  coordinates  is  hidden  from 
the  user.  It  is  fairly  easy  for  the  user  to  define  new  types  of  plots  by 
creating  new  hierarchies  of  Plot-Objects. 


1.1.3  Interactive,  Motion  Graphics 

The  current  design  for  interactive  graphics  is  fairly  tentative.  Implementa¬ 
tion  of  a  portable  scientific  graphics  toolkit  requires  a  standardized  interface 
between  Common  Lisp/CLOS  and  the  large  variety  of  proprietary  or  pro¬ 
posed  standard  window  systems  for  workstations  and  personal  computers 
(eg.  Symbolics  Genera  [25],  NeWS[24],  X[20],  etc.).  This  standard  (some¬ 
times  called  Conmon  Windows)  is  the  subject  of  intense  activity  in  the 
Common  Lisp  commurity[9,  18|.  2D-Graphics/2D-PIot  is  our  current  best 
approximation;  it  will  be  modified  or  will  wither  away  as  a  true  Common 
Windows  standard  emerges. 

The  parts  of  2D-Graphic8  and  2D-Plot  that  are  not  specific  to  a  particu¬ 
lar  display  device  were  implemented  in  pure  Common  Lisp,  for  portablility. 
Since  the  CLOS  standard  is  now  more  or  less  fixed,  we  will  convert  them  to 
use  it,  as  a  first  step  in  supporting  interactive  graphics. 

I  have  identified  three  modules  in  a  future  interactive  graphics  subsys¬ 
tem: 

•  Constraints,  which  requires  Common  Lisp  and  CLOS.  (This  module 
might  very  well  be  part  of  the  non-graphical  kernel,  but  most  of  the 
applications  we  have  in  mind  at  present  are  in  graphics.) 

•  Quantitative  Graphics,  which  requires  Common  Windows,  Collections, 
Constraints,  and  Linear  Algebra. 

•  Data  Analysis  Graphics,  which  requires  Quantitative  Graphics  and 
Statistics 

The  contents  of  these  modules  are  described  in  [13], 
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1.2  Cactus 


I 


The  underlying  premise  of  much  of  my  research  is  that  quantitative,  scientific 
computing  needs  and  deserves  good  programming  languages  and  program¬ 
ming  environments — as  much  as  artificial  intelligence.  The  Cactus  system 
shows  how  straightforward  application  of  object-oriented  design  to  stan¬ 
dard  algorithms  in  numerical  analysis  yields  immense  improvement  in  clar¬ 
ity,  without  sacrificing  speed  or  numerical  accuracy. 

Object-oriented  programming  is  sometimes  said  to  only  be  useful  for 
graphics  and  user  interface  and,  perhaps,  knowledge  representation  and 
database  management.  Even  Lisp  machine  manufacturers  seem  to  think 
that  Fortran  is  somehow  superior  for  traditional,  quantitative  scientific  com¬ 
puting.  When  numerical  software  is  written  in  Lisp  (eg.  [19]),  the  author 
usually  adopts  a  Fortran  or  Algol  style  and  neglects  the  potential  for  de¬ 
signing  more  appropriate  abstractions[l]. 

Cactus  is  based  on  a  collection  of  linear  transformation  classes  and  appro¬ 
priate  generic  operations.  This  level  of  abstraction  greatly  simplifies  many 
algorithms  in  numerical  linear  algebra.  Traditional  linear  algebra  systems 
(Linpack[6],  APL)  operate  at  the  level  of  arrays  and  confound  the  detaib  of 
where  data  is  kept  with  how  it  is  meant  to  be  used. 


1.3  Fossil 

I  organized  a  small  meeting  in  Seattle  in  November,  1986,  which  was  at¬ 
tended  by: 

•  Keith  Kerr,  Applied  Physics  Laboratory,  U.  Washington 

•  John  McDonald,  Statistics,  U.  Washington 

•  John  Michalak,  Statistics,  U.  Washington 

•  Wayne  Oldford,  U.  Waterloo 

•  Jan  Pedersen,  Xerox  and  Stanford 

•  Stephen  C.  Peters,  U.C.  Berkeley 
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•  Weraer  Stuet.zle,  Statistics,  U.  Washington 

•  Alan  Wilks,  AT&T  Bell  Labs. 


The  purpose  of  the  meeting  was  to  collect  together  a  group  of  people 
interested  in  statistical  computing  in  Lisp.  We  intended  to  collaborate  in  an 
informal  joint  project  called  Fossil.  The  primary  goal  of  Fossil  was  to  build 
up  a  portable  library  of  scientific  subroutines — statistical,  numerical,  and 
graphics  functions — that  can  be  shared  by  people  working  in  any  implemen¬ 
tation  of  Common  Lisp.  This  would  eliminate  a  present  tendency  towards 
re-inventing  the  wheel  among  statistics  researchers  using  Common  Lisp.  To 
be  able  to  share  software,  we  needed  to  agree  on  a  number  of  things,  in 
particular,  consistant  data  structures  and  function  libraries. 

The  most  direct,  concrete  result  of  the  Fossil  meeting  was  Polish-Fortran, 
a  portable  Common  Lisp  program  that  translates  Fortran  source  code  into 
a  Lisp-like  syntax  (polish  notation).  A  set  of  portable  Common  Lisp  macros 
are  provided  that  simulate  Fortran  semantics.  Although  some  Lisp  envi¬ 
ronments  provide  mechanisms  for  calling  Fortran  subroutines,  these  foreign 
function  Interfaces  are  not  standardized  and,  therefore,  code  that  relies  on 
them  is  not  portable.  In  addition,  the  translator  can  and  has  been  used 
as  a  first  step  in  translating  Fortran  into  readable,  modifiable,  extensible 
Common  Lisp  code.  This  was  done  for  some  of  the  special  functions  in  the 
Basic  Math  module  in  Arizona. 


1.4  Antelope 

One  part  of  my  research  was  devoted  to  two  related,  “high  level”  topics: 
object-oriented  representation  of  statistical  data  and  Object-viewing,  a  con¬ 
straint  approach  to  data  analysis  graphics.  The  result  of  this  work  is  an 
experimental  system  called  Antelope,  which  is  described  in  [11], 

An  important  part  of  Antelope  was  the  Antelope  language.  I  extended 
the  object-oriented  Flavors  language  provided  on  the  Symbolics,  because 
it  did  have  all  the  properties  necessary  to  support  the  statistical  object- 
oriented  databases  or  constraint-based  graphics.  The  Antelope  language 
is  now  unnecessary,  because  CLOS  supplies  some  of  the  features  that  were 
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oiissing  in  Flavors,  and  provides  a  systematic  and  portable  way  to  add  others 
via  the  meta-object  protocol. 

In  Antelope,  statistical  data  is  represented  by  objects,  the  primitive  data 
structures  in  the  object-oriented  Antelope  language.  The  choice  to  represent 
statistical  data  directly  in  the  most  primitive  data  structures  of  the  program¬ 
ming  language,  rather  than  adding  a  more  elaborate  statistical  database, 
is  both  minimal  and  harmonious  with  the  spirit  of  base  environment.  The 
relevant  point  here  is  that  statistical  data  objects  do  not  differ  in  any  funda¬ 
mental  way  from  other  objects  in  the  programming  environment.  Statistical 
tools  can  be  applied  to  any  objects  (for  example,  the  windows  on  the  screen 
or  the  processes  managed  by  the  scheduler);  the  regular  programming  tools 
(for  example,  the  editor  or  the  inspector)  can  be  used  on  statistical  objects. 

The  constraint  paradigm  for  graphics  is  used  to  support  Object-viewing, 
which  means  that  windows  are  thought  of  as  views  of  the  current  state  of 
objects  in  the  environment.  For  example,  scatterplot  windows  are  graphical 
views  of  data  sets;  editor  windows  are  text  views  of  procedure  objects.  The 
key  point  is  that  windows  automatically  update  their  appearance  whenever 
the  objects  they  show  change  state.  If  a  record  object  has  a  slot  called  color, 
and  the  value  of  color  is  changed  from  red  to  green,  then  all  windows  that 
show  some  representation  of  that  particular  record  will  change  the  color 
from  red  to  green.  The  implementor  of  code  which  modifies  a  record  object 
does  not  have  to  worry  about  whether  the  record  is  shown  in  one  or  more 
windows  and  that  these  windows  need  to  be  updated  whenever  the  record 
changes. 

In  contrast,  most  existing  window  systems  treat  windows  as  virtual  paper 
and  their  contents  as  virtual  ink.  Drawing  on  a  window  makes  a  mark 
that  remains  until  it  is  explicitly  erased  and  re-drawn.  One  motivation 
for  a  systematic  implementation  of  object-viewing  is  the  belief  that  most 
erasing  and  redrawing  of  windows  is  done  to  reflect  a  change  of  state  of 
some  object  in  the  environment.  Thus  common  tasks  in  interactive  graphics 
are  simplified  if  erasing  and  redrawing  can  be  done  automatically. 

Object-viewing  is  best  thought  of  as  imposing  a  constraint  between  the 
abstract  object  in  the  programming  environment  and  its  visible  representa¬ 
tion  on  the  screen.  A  good  constraint  language  gives  the  programmer  clear, 
simple  ways  to  express  constraints  and  also  encourages  a  modular  design — 
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by  separating  the  description  and  implementation  of  constraints  from  the 
implementation  of  the  internal  details  of  the  constrained  objects. 


In  a  little  more  detail,  the  object-viewing  programming  model  consists 
of: 

•  A  number  of  base  objects  residing  in  the  programming  environment’s 
address  space,  like  a  record  in  a  statistical  database  or  the  definition 
of  a  Lisp  function. 

•  Visible  representations  of  base  objects  in  windows.  Abstractly,  the 
visible  representation  corresponds  to  something  like  a  node  of  a  hier¬ 
archical  display  tree. 

•  A  viewing  filter  (or  viewing  pipeline  [5])  that  computes  the  state  of  the 
visible  representation  from  the  state  of  the  base  object.  An  obvious 
example  of  a  viewing  filter  is  the  usual  3-d  viewing  transformation, 
which  might  be  used  is  a  rotating  scatterplot.  A  less  obvious  example 
is  a  pretty-printer  that  is  applied  to  the  definition  of  a  Lisp  function 
before  it  is  displayed  in  an  editor  window. 

•  A  constraint  that  causes  the  visible  representation  to  be  re-computed 
(and  re-drawn)  whenever  the  state  of  the  base  object  changes. 

Further  applications  of  the  constraint  paradigm  to  statistical  graphics 
are  described  in  [4,  5,  8]. 


2  Professional  Activity 

2.1  Refereed  Publications: 

1987  Computing  Environments  for  Data  Analysis,  Part  HI:  Programming 
Environments,  with  Jan  Pedersen,  SIAM  J.  Scientific  and  Statistical 
Computing  9  (2):  380-400. 

1986  Smoothing  with  Split  Linear  Fits,  with  Art  Owen,  Technometrics  28 
(3):  195-208. 

1986  Periodic  Smoothing  of  Time  Series,  SIAM  J.  Scientific  and  Statistical 
Computing  7  (2):  665-688. 
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2.2  Invited  Papers  in  Conference  Proceedings; 

1988  An  outline  of  Arizona:  a  portable  Lisp-based  system  for  data  analysis, 
at  Computer  Science  and  Statistics,  the  20th  Symposium  on  the  In¬ 
terface,  Reston,  Va.,  April.  (Also  Tech  Rept  131,  Dept,  of  Statistics, 
U.  of  Washington) 

1986  Antelope:  data  analysis  with  object-oriented  programming  and  con¬ 
straints,  Proc.  of  the  1986  Joint  Statistical  Meetings,  Stat.  Comp.  Sect, 
(also  Tech  Rept  89,  Dept,  of  Statistics,  U.  of  Washington). 

2.3  Other  Publications: 

1988  Elements  of  a  viewing  pipeline  for  data  analysis,  with  Andreas  Buja, 
Daniel  Asimov,  and  Catherine  Hurley,  in  Dynamic  Graphics  for  Statis¬ 
tics,  Cleveland,  W.S.,  and  McGill,  M.E.,  eds.  (1988)  Wadsworth,  Pa¬ 
cific  Grove,  Ca. 

1988  Interactive  Graphics  for  Data  Analysis,  (Ph.D.  Thesis)  Available  as 
Tech.  Report  Orion#  11,  Dept,  of  Statistics,  Stanford  University. 
Also  in  Dynamic  Graphics  for  Statistics,  Cleveland,  W.S.,  and  McGill, 
M.E.,  eds.  (1988)  Wadsworth,  Pacific  Grove,  Ca. 

1987  Object-oriented  design  in  numerical  linear  algebra,  Technical  Report 
109,  Dept,  of  Statistics,  U.  of  Washington. 


2.4  Invited  Presentations; 

1988  Analysis  of  fish  abundance  in  the  Bering  sea:  a  case  study  in  the  use 
of  graphical  methods,  with  Werner  Stuetzle,  at  the  Joint  Statistical 
Meetings  in  New  Orleans  in  August  1988. 

1988  An  outline  of  Arizona:  a  portable  Lisp-based  system  for  data  analy¬ 
sis,  at  Computer  Science  and  Statistics,  the  20th  Symposium  on  the 
Interface,  Reston,  Va.,  April. 

1987  Object-oriented  design  in  numerical  linear  algebra,  at  Computer  Sci¬ 
ence  and  Statistics,  the  I9th  Symposium  on  the  Interface,  Philadel¬ 
phia,  March. 
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1986  Antelope:  data  analysis  with  object-oriented  programming  and  con¬ 
straints,  Proc.  of  the  1986  Joint  Statistical  Meetings,  Stat.  Cornp.  Sect. 

1980  The  object-viewer  paradigm  for  data  analysis  graphics  at  the  AMS 
meeting  on  Data  Analysis  and  Graphics  in  Santa  Cruz  in  June. 

1986  The  object-viewer  pamdigm  for  data  analysis  graphics  at  the  session 
on  Graphical  Components  for  Statistical  Workstations  of  the  National 
Computer  Graphics  Association  meeting  in  May. 


I  have  given  invited  seminars  in  the  Statistics  Dept.,  Computer  Science 
Dept.,  the  Applied  Physics  Lab  at  the  U.  of  Washington,  at  Stanford,  MIT, 
U.  of  Waterloo,  and  F'lorida  State,  and  in  research  labs  at  AT&T  Bell  Labs, 
Bellcore,  Schlumberger-DoU  Research,  Schlumberger  Palo  Alto,  and  Xerox 
PARC. 

In  addition,  the  ONR  Young  Investigator  Award  has  supported  my  at¬ 
tendance  at  a  number  of  conferences  where  I  did  not  give  presentations:  the 
Common  Lisp  Object  System  Workshop  at  Xerox  PARC  in  October  1988, 
the  2nd  OOPSLA  (Object-Oriented  Programming:  Systems,  Languages, 
and  Applications)  meeting  in  Orlando  in  October  1987,  the  Symbolics  Na¬ 
tional  Users  Group  meeting  in  Seattle  in  July  1987,  the  AAAI  meeting  in 
Seattle  in  July  1987,  the  Joint  Statistical  Meetings  in  San  Francisco  in  Au¬ 
gust  1987,  and  the  1st  OOPSLA  meeting  in  Portland  in  October  1986. 

2.5  Graduate  Students 

•  Ph.D.  committee  for  Catherine  Hurley.  Degree  received  Fall  1987. 
Thesis  title:  The  Data  Viewer:  A  program  for  graphical  data  analysis. 

•  Ph.D.  committee  for  Jeff  Banfield.  Degree  received  spring  1988.  Thesis 
topic:  Clustering  and  image  processing  with  applications  to  remote 
sensing  of  sea  ice. 

•  Ph.D.  committee  for  Pat  Burns.  Degree  received  in  Jan  1988.  Thesis 
topic:  Resistant  Fits  of  Two-way  tables. 

•  Ph.D.  committee  for  John  Michalak.  Degree  to  be  received  spring 
1989.  Thesis  topic:  Interactive  methods  for  viewing  hierarchical  struc¬ 
tures  in  data  analysis. 
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•  Ph.D.  committee  for  Charlie  Geyer.  Thesis  topic:  Constrained  Maxi¬ 
mum  Likelihood  for  exponential  families. 

In  addition,  I  informally  advise  other  Statistics,  Biostatistics,  and  Com¬ 
puter  Science  Ph.D.  students  who  use  our  Lisp  machines  in  their  research, 
including  Deborah  Donnell,  Andrew  Bruce,  Robert  Gentleman,  Mike  Kahn, 
Jorean  Sicks,  Steve  McKinney  (Biostat),  Kevin  Anderson  (Bioetat),  Jed 
Dennis,  Suzanne  Weghorst  (CS),  and  Michael  Sannella  (CS). 
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